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Systems Engineering Metrics: 

Organizational Complexity and Product Quality Modeling 


Executive Summary 

Innovative organizational complexity and product quality models applicable to performance 
metrics for NASA-MSFC’s Systems Analysis and Integration Laboratory (SAIL) missions and 
objectives are presented An intensive research effort focuses on the synergistic combination of 
stochastic process modeling, nodal and spatial decomposition techniques, organizational and 
computational complexity, systems science and metrics, chaos, and proprietary statistical tools for 
accelerated risk assessment. This is followed by the development of a preliminary model, which is 
uniquely applicable and robust for quantitative purposes. Exercise of the preliminary model using 
a generic system hierarchy and the AXAF-I architectural hierarchy is provided. The Kendall test 
for positive dependence provides an initial verification and validation of the model. Finally, the 
research and development of the innovation is revisited, prior to peer review. This research and 
development effort results in near-term, measurable SAIL organizational and product quality 
methodologies, enhanced organizational risk assessment and evolutionary modeling results, and 
improved statistical quantification of SAIL productivity interests. 

Major conclusions include: 

1. The model provides programmatic risk as a complex, nonlinear function of relative 
architectural complexity, organizational complexity, budgetary weightings, technological 
maturities, and schedule duration. 

2. The model form is scale-invariant (geometric), hierarchically recursive, and evolutionary, with 
a strong engineering foundation, and supported by current complexity and metamodel research. 

3. Preliminary expansion/contraction rules have been developed for assessing the impact of 
budgetary and schedule changes on programmatic risk. 

4. Preliminary sensitivity analyses indicate budget contractions are more damaging to risk than 
schedule contractions, while schedule expansions are more helpful than budgetary ones. 

5. Major programmatic risk drivers for the model analysis of AXAF-I showed a positive 
dependence of 88% with that of AXAF-I program management. 

6. Major model uncertainty drivers, in descending order, are basis function parameters, 
expansion/contraction rulebase, and organizational risk factors. 
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Key recommendations are: 

1 . Investigate the model validity, focusing on the uncertainty drivers of conclusion 6. 

2. Utilize advanced statistical techniques to maximize validation information from limited data 
sources. 

3. Incorporate a quality monitoring system for management control and feedback purposes. 

4. Investigate model control mechanisms to achieve minimum risk for a given programmatic 
payoff. 
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Systems Engineering Metrics: 
Organizational Complexity and Product Quality Modeling 


1.0 Introduction 

The field of Systems Science has enjoyed a resurgence as organizations have come to terms with 
shrinking budgets, aging resources, and increasing and more time critical requirements. The 
current strategic conditions are particularly distressing for larger, more bureaucratic 
organizations, such as public agencies, since the decentralized nature of these organizations, the 
very characteristic that provides the mechanism for superior technical performance, is a liability to 
the requirement of rapid change. Public agencies most susceptible to these conditions, currently, 
are large R&D organizations of a highly specialized nature. NASA-MSFC falls into this category, 
with its additional pressures of safety, schedule, and large complex projects. Large R&D 
organizations suffer from an additional difficulty when attempting to deal with these stressing 
requirements: how to measure productivity and quality in a meaningful way for R&D and 

intellectual products. NASA-MSFC’s Systems Analysis and Integration (SAIL) Laboratory is 
responsible for providing some of the answers to this dilemma for MSFC, but also for its own 
organization. 

As a consequence of the difficulty of measuring quality and productivity characteristics for 
intellectual and R&D products, most attempts at quantification are very sensitive to the method of 
quantification, as well as the assumptions required to carry out the analysis. This lack of 
robustness is particularly disturbing, since it results in the well-known phenomena of “tweaking 
the model parameters until the desired answer is achieved.” This tendency is dangerous, in that it 
can result in inappropriately shoring of the status quo, or effecting rapid change that is damaging 
to the organizational missions. 

A second problem with modeling organizational productivity and quality is the static nature of 
most attempts at quantification. Most modeling attempts do not account for the dynamic, 
evolutionary nature of organizations in times of rapid change. Consequently, the models either 
have to be frequently updated, or changed completely, an expensive proposition. 

Lloyd [1996] has described a complex situation as one where despite perfect understanding of the 
subsystems, the integrated system behavior is not well understood. He has emphasized the 
question: ”In general, how do complex, specific laws arise from simple, generic ones?” 

A number of attempts have been made at developing robust, dynamical models of R&D 
organizational effectiveness. Streufert and Swezey [1986] have applied complexity theory to 
achieve measurements via a time-event matrix. Klir [1985] has also pursued complexity 
contributions combined with a metasystems approach to deliver robust results. Cameron and 
Whetten [1983] have addressed the demise of organizational effectiveness studies, and 
investigated the implications for effectiveness theory in the public sector. 
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Study Objectives 

The overriding objective of this study is to produce an intensive research-based, peer review 
acceptable, robust, and quantitative systems meta-model for the SAIL organizational mission and 
products. Particular emphasis will be placed on providing a model which is relatively insensitive 
to tuning parameters and assumptions (robust), and evolutionary (dynamical) in nature. The three 
main questions to answer in this study are: 

1. What is a mathematical relation of system architectural complexity to system size and the 
number of intersystem and intrasystem interfaces? 

2 What is a mathematical relation of system complexity to system size, technological maturity, 
and the number of intersystem and intrasystem interfaces 9 

3. What is a mathematical relation of project complexity to system size, technological maturity, 
organizational dispersion, and the number of intersystem and intrasystem interfaces? 

The answers to these three questions, as determined by this study are. 

1 . The architectural complexity of the system is proportional to an enumeration of the number of 
hierarchical levels and the number of interactions amongst systems and subsystems The 
maximum number of interactions at a given subsystem level is equal to one-half of the product of 
the number of components and the number of components minus one. (See Figure 2.0-1 .) 

2. System complexity is proportional to an enumeration of the minimum number of parameters 
required to describe system size, technological maturity, and the number of interactions. System 
size is defined by the number of hierarchical levels and the number of subsystems at each level. 
The number of interactions is described above. Technological maturity is a function of time, as a 
minimum. (See Figure 2.0-1. and Section 4.0) 

3. Project complexity is proportional to an enumeration of the minimum number of parameters 
required to describe system size, technological maturity, organizational dispersion, and the 
number of interactions. System size is defined by the number of hierarchical levels and the 
number of subsystems at each level. The number of interactions is described above. 
Technological maturity is a function of time, as a minimum. Organizational dispersion is a 
function of organization size, number of hierarchical levels, and number of interactions amongst 
individuals at each level. (See Figure 2.0-1. and Section 4.0) 


2.0 Complexity Measures 

(Schuster, 1996) explains how nature employs optimization during times of scarcity, creative 
innovation in times of abundance, and modular design to control uncertainty over time. While 
nature invariably uses increasing levels of complexity in successive stages of evolution, the 
manner in which complexity is employed is neither random nor linear; rather, nature employs 
complexity prudently. 
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Project success commonly has been defined as completion of an activity within time, cost, and 
performance constraints (Kerzner, 1989). To be relevant within the context of engineering 
systems development, a metric for system complexity must be comprehensible in terms of system 
technical performance, development cost, and schedule. Hence, the prudence of increased 
complexity in the engineering arena is determined on the basis of technical performance, 
development cost, and schedule. More specifically, since technical performance, development 
cost, and schedule are effectively joint stochastic variables prior to project completion, the 
prudence of increased complexity may be assessed in terms of the risk associated with each. 

A system complexity metric could then be of value in analyzing the inherent difficulty associated 
with the project plan to build a system. It could be used in at least the following three manners: 

• Identify key components within the system architecture, whose developmental problems will 
ripple widely in the system development process 

• Identify locales within the system architecture where higher technology readiness is needed 
and others where lower technology readiness can be tolerated. 

• Identify organizational bottlenecks for a given organizational scheme to build the system. 

In each of these three areas, the inherent difficulty is measured in terms of risk associated with the 
project technical performance, cost, and schedule. 


We conjecture, intuitively, that the complexity of a system is a function of the technological 
maturity of its components, and the organizational and technological dispersion associated with 
developing the system. In turn, both the technological maturity and organizational/technological 
dispersion are functions of the size of the system and the development duration of the schedule. 
Casti [1994] states that “the complexity is directly proportional to the length of the shortest 
possible description of that object.” Thus, system complexity is proportional to a count on the 
description parameters associated with the system size, technological maturity, organizational and 
technological dispersions, and development horizon. Finally, the maximum number of subsystem 
or component interactions in a two dimensional hierarchy is one-half the product of the system 
size times the system size - 1. These initial thoughts regarding complexity measures are provided 
in mathematical form in Figure 2.0-1 below. 

Here, C is the system complexity measure, X is the system technological maturity, a is the 
combined organizational/technological dispersion, n is the system size (usually, the number of 
components or subsystems), t is the development time or duration, and M is the maximum number 
of subsystem interactions. 
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Figure 2.0-1. Initial Mathematical Thoughts on Complexity 


3.0 Preliminary Model Form 

The preliminary complexity model form is shown in Figure 3.0-1. This form is a geometric or 
power form common to engineering analysis. According to Hastings and Sugihara [1995], the 
geometric form is the only form which is scale-invariant. This is important, because the model’s 
utility is determined by the number of different systems (of different scales) to which it is 
applicable. This model form allows both independent and interacting subsystems in an 
evolutionary and recursive hierarchy. Handling both independent and interactive subsystems 
ensures a broader range of application, while recursiveness is desirable to address systems with 
differing numbers of layers in the hierarchy The model form is also an efficient and commonly 
used data-fitting form, important for model validation. Finally, the model form allows for the use 
of neural net and global optimization solvers, an important consideration when the issue of system 
control is addressed. Due to these desirable attributes, this model form is found frequently in 
complexity, fractal, and metamodel literature. 
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Figure 3.0-1. Preliminary Complexity Model Form 
4.0 Preliminary Complexity Model 

The preliminary complexity model must be dependent on more than the size of the system. It 
should also take into account schedule and budgetary issues, technology choice and relative 
investment, the system architectural design, and relevant organizational considerations. The 
preliminary model. Complex Organizational Metric for Programmatic Risk Environments 
(COMPRE), is given by: 
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where 


X(t) is a (time-dependent) technology development maturity function, representing a rate of 
payoff for investing in a certain level of technology, 

n is the number of technology investments made, 

Wi (t) is the relative investment weight for technology i, 

E[r, (t)] is the expected retum-on-investment for technology i over the development schedule 
represented by t, 

oft) represents total programmatic risk, and 

Cij (t) is the covariance between technologies i and j over the development duration. 

The development maturity function, X, accounts for technology, schedule, and budgetary 
distribution decisions, as well as system size. The programmatic risk function, o, captures the 
same issues, as well as architectural design and organizational makeup. Preliminary results for the 
AXAF-I (NASA’s Advanced X-ray Astrophysics Observatory) project, portfolio management 
problems, and computer system replacement decisions indicate that the ratio of these two 
functions may provide a useful complexity measure for decision-making and control purposes. 


5.0 Standardized Complexity Methodology (COMPRE) 


Figure 5.0-1 shows the standard methodology using COMPRE. Given a system architecture 
which includes technology readiness and organizational data as depicted in Figure 5.0-2, the 
profiles for the relative technology investments are captured in a component influence vector. 
Engineering basis functions (typically, linear, quadratic, and exponential) are utilized to develop 
the profile for a technology evolution vector. Once the development schedule is identified, the 
programmatic payoff can be determined as a function of the schedule. A technology covariance 
matrix is the intermediate step to achieving the programmatic risk profile. This matrix is adjusted 
for the presence, absence, and degree of architectural and organizational interactions. 

The raw data required to run COMPRE are: 

1. A project architecture/hierarchy, showing all systems, subsystems, components, and 
interactions at all levels. 

2 The technological maturities for all components of the system at the lowest levels of the 
hierarchy 

3. Organizational responsibility for each of the systems, subsystems, and components of the 
hierarchy. 
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4. Relative budgetary investments in each of the systems, subsystems, and components of the 
hierarchy. 

5. The schedule duration of the project development. 

In addition to these five data requirements, COMPRE uses the following data, models, and 
assumptions: 

1 . Technology basis function models to describe technological maturity. 

2. A rulebase for organizational dispersion factors (between organization dispersion). 

3. A rulebase for budgetary/schedule contractions and expansions for conducting relative 
sensitivity analyses 

Appendix B provides numerical details, relating to these data and assumptions for the AXAF-I 
example of Section 7.0. 
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Figure 5.0-1. Standardized COMPRE Methodology 
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The resulting complexity measure is then a function of cost, schedule, technology, architecture, 
and organization. When the measure is the ratio of programmatic risk to payoff, an understanding 
emerges concerning the efficacy with which the organization, architecture, technology, and 
resources (both time and dollars) are being utilized. Furthermore, this ratio may be used to 
identify those individual subsystems and architectural interactions that are contributing the most 
risk, relative to payoff, for the overall system being investigated. 


6.0 Generic COMPRE Sensitivity Analysis 

To gain a basic understanding of the performance of COMPRE, a generic sensitivity analysis is 
performed. The baseline assumptions for this sensitivity analysis are as follows: 

1. This is a single layer hierarchy with three components, or subsystems, each interacting with 
each other 

2. A single organization is responsible for the development of all three components. 

3. The budget is allocated uniformly across the three components. 

4. The technology maturity classification is 3 (some new technology) for all three components. 

5. The development duration is 5 time units. 

Moreover, the following expansion/contraction rulebase is used to implement schedule and 
budgetary changes from the baseline: 

Al. A budget change results in a single component technological maturity change, in addition to 
the reallocation of the relative component budgets. 

A2. A schedule duration increase results in all components experiencing a maturity improvement, 
in addition to the change experienced by expansion of the program development. 

A3. A schedule duration contraction results in all components experiencing a maturity 
degradation, and an increase in the number of organizations required to carry out the 
development, in addition to the change experienced by contraction of the program development. 

Figure 6.0-1 shows the results of the generic sensitivity analysis. The total relative programmatic 
risk (risk divided by baseline risk) is plotted for 4 cases plus the baseline. This sensitivity shows 
that schedule duration increases are more effective than budgetary ones in reducing programmatic 
risk. However, budgetary reductions are slightly more damaging than schedule contractions in 
terms of risk. This is because shortening the schedule reduces the inherent volatility in program 
development, somewhat offsetting the increased risk of less mature technology. 
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Figure 6.0-1. Generic System Sensitivity Analysis 


7.0 AXAF-1 Analysis Using COMPRE 

In this section, a COMPRE analysis is performed for the AXAF-I program. Figure 7.0-1 shows 
the architectural hierarchy (nodal decomposition) for AXAF-I. AXAF-I is composed of three 
major subsystems: Spacecraft, Telescope, and ISIM. These three subsystems are further 
decomposed into their respective components. The numbers next to the subsystems and 
components represent the development maturities for the component technologies. The code for 
these maturities is found in the upper left-hand comer of the figure. Green lines connecting 
subsystems and components represent interfaces or interactions. Colors and shading represent 
organizational responsibilities via the organization codes located at the bottom right-hand comer 
of the figure. External interfaces are represented by magenta lines, but are not included in the 
analysis. Although development maturity codes are provided as estimates for the Spacecraft, 
Telescope, and ISIM subsystems, COMPRE derives these values itself. 

Figure 7.0-2 shows the COMPRE plot for AXAF-I. This plot provides the coordinates for the 
expected technological maturity, X, and the expected programmatic risk, a, given by the 
preliminary (COMPRE) model for a duration of 5 years, ending in 1998. These values are based 
on the relative budget weights given in Table 7.0-1, below. Clearly, the Telescope and ISIM 
subsystems, packed with newer technology, pose the greatest risk to the program. The 
Spacecraft subsystem, meanwhile, carries much less programmatic risk. Finally, the properties of 
the AXAF-I system, as a whole, are properly weighted averages of the major subsystems. 
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AXAF-I Complexity Analysis (1998) 
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Figure 7.0-2. AXAF-I COMPRE Plot 
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Figure 7.0-3 shows the normalized COMPRE risk, a/A., for the three major AXAF-I subsystems. 
In both the normalized and absolute risk cases, the ISIM and Telescope subsystems dwarf the 
Spacecraft subsystem. However, the ISIM and Telescope have exchanged places in rank, 
indicating that for a constant expected technological payoff, the ISIM carries more risk than the 
Telescope. This result is consistent with the rankings provided by AXAF-I Program 
Management, who ranked ISIM problems as their top four out of five (see below). 

Figure 7.0-4 shows the major risk sources, as provided by COMPRE. Two of the top four 
involved the HRMA, and the other two involved the ACIS. Figure 7.0-5 shows the major 
normalized risk sources, as provided by COMPRE. The top two sources involve the SIM 
interactions, ranked third and fourth by AXAF-I management. 

Table 7.0-2 provides the rankings of the AXAF-I Program Management, and the correlation with 
COMPRE programmatic risk and normalized risk. Confidence values, given by Kendall’s 
measure of association are given. The confidence values should be interpreted as follows: there 
is an 88% chance that the rankings provided by AXAF-I Program Management and COMPRE’s 
programmatic risk are positively dependent. Similarly, there is an 59% chance that the rankings 
provided by AXAF-I Program Management and COMPRE’s normalized risk are positively 
dependent. It is possible that both of these results are better than expected at this stage of 
COMPRE development, given the fact that the COMPRE basis functions have not been baselined 
or validated. However, the difference in confidence values between the absolute and normalized 
risks should not be overemphasized at this juncture. Recall that at the subsystem level, COMPRE 
reversed the ISIM and Telescope rankings using normalized risk, correctly (with respect to 
AXAF-I Program Management) emphasizing the ISIM over the Telescope. The normalized risk 
underperformance at the component level could be a matter of COMPRE refinement. 
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Figure 7.0-5. Major AXAF-I Normalized Risk Sources 


Table 7.0-2. Correlated Rankings: AXAF-I Program Management and COMPRE 


Component/ 

Interaction 

Program 

Management 

Ranking 

COMPRE 

Programmatic 

Risk 

COMPRE 

Normalized 

Risk 

ACIS 

1 

2 

6 

HRC 

2 

>7 

>7 

ACIS-SIM 

3 

3 

1 

HRC-SIM 

4 ^ 

7 H 

2 

Propulsion 

5 

>7 and 
>HRC 

>7 and 
>HRC 

Confidence 


88% 

59% 


17 


















A sensitivity analysis was performed for AXAF-I budgetary and schedule expansions and 
contractions. The expansion contraction rulebase for budget changes is as follows: 

B 1 . A +5% budget change allows for a single component technology maturity change from a 1 to 
0 or 2 to 1, and forces a reallocation of the budget toward that component. 

B2. A +10% budget change allows for a single component technology maturity change from a 4 
to 3 or 3 to 2, and forces a reallocation of the budget toward that component 

B3. A +15% budget change allows for a single component technology maturity change from a 5 
to 4 or removal of one interaction 

B4. A +20% budget change allows for a single component technology maturity upgrade of two 
levels or removal of 2 interactions or two upgrades of one level each. 

Budget reductions work the same, but by reversing the direction from upgrade to downgrade, or 
by adding interactions. 

The schedule change rulebase is given by the following: 

Cl. A +5% schedule expansion allows for a single component technology maturity change from a 
1 to 0 or 2 to 1 . 

C2. A +10% schedule expansion allows for a single component technology maturity change from 
a 4 to 3 or 3 to 2. 

C3. A +15% schedule expansion allows for a single component technology maturity change from 
a 5 to 4 or removal of one interaction. 

C4. A +20% schedule expansion allows for a single component technology maturity upgrade of 
two levels or removal of 2 interactions or two upgrades of one level each. 

Schedule contractions work the same, but by reversing the direction from upgrade to downgrade, 
or by adding interactions. 


Figure 7.0-6 shows the sensitivity analysis for the schedule changes. The % baseline (100% 
budget) programmatic risk is shown versus the % baseline budget. It is interesting to note the 
lack of symmetry near the baseline budget (100%). Programmatic risk is more sensitive to budget 
decreases than increases near the baseline. This is because small increases in budget do not 
significantly alter the technological maturity of a single component, but they do overweight that 
component’s relative performance. Thus, little risk is initially reduced. On the other side, a small 
budget contraction can tilt a program’s delicate balance. Note that large changes, whether 
positive or negative in nature, do result in approximately symmetric sensitivities. 

Figure 7.0-7 shows the AXAF-I/COMPRE surface for the relative impact of changes in budget 
and schedule on programmatic risk. While programmatic risk is generally monotonically 
decreasing with increasing budget allocations (applied judiciously, of course), the sensitivity of 
risk to changes in schedule duration is more complex. Above the baseline schedule duration. 
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reductions in technological risk more than compensate for the expected increases in system 
variability associated with a longer development timeframe. Thus, the overall programmatic risk 
is mostly reduced with increasing schedule durations. Below the baseline duration, however, the 
overall programmatic risk decreases with tightening schedules. This is because increasing 
technological risk is more than compensated for by decreasing system variability associated with 
tighter schedules Thus, an AXAF-I risk containment strategy can actually be achieved by a 
tightening of the schedule. 



Figure 7.0-6. AXAF-I Budgetary/Contraction Sensitivity 
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Figure 7.0-7. Representative COMPRE Risk Sensitivity Surface 
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8.0 Preliminary COMPRE Verification and Validation 

Currently, COMPRE model validation is accomplished by performing statistical hypothesis tests 
on the degree of association between COMPRE’s ranking of risk contributors and the ranking of 
“hot spots” as noted by program management. Typical association measures by Kendall and 
Spearman are easily understood and provide robust, nonparametric conclusions. COMPRE 
validation plans include the analysis of both governmental and commercial programs. 

A method for validating the Type I and II error rates in COMPRE is currently planned. The Type 
I error probability is the probability that the COMPRE model is concluded to have 
underperformed its benchmark, when it actually outperformed it. The Type II error probability is 
the probability that the COMPRE model is concluded to have outperformed its benchmark, when 
it actually underperformed it. Acceptable benchmarks include Kendall confidence levels, as well 
as other statistical measures. Acceptable Type II error rates are usually smaller than acceptable 
Type I error rates, which are often in the 10-20% range for most types of risk assessments. OR 
Applications has developed a proprietary statistical acceleration technique for maximizing the 
statistical information for a limited amount of validation data. This technique is associated with 
the general field of statistical resampling, the method of calculation shown below. 


a = p { ^ 


o «, 


P- > PZJ^ ) 


fi=pt p \ - f° ) 

< 7 , <7 


where 

a = the resampled estimate for the probability of Type I error, 

/? = the resampled estimate for the probability of Type II error, 
p. = the resampled estimate for the performance measure, 
p = the data estimate for the performance measure, 
p 0 = the benchmark performance measure, 

<x, - the resampled estimate for the standard deviation of the data, 
b = the data estimate for the standard deviation of the data. 


9.0 Quality Modeling Using COMPRE 

A concept for monitoring and quality control of COMPRE is also being developed. This concept 
is particularly important as COMPRE moves from a predictive stage to a prescriptive one. 
Control of COMPRE takes many forms: schedule and budgetary contractions and expansions, 
technology choice, investment weighting decisions; architectural design and modification as the 
program unfolds; and organizational makeup and design for implementation purposes. 
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Each of these decisions permeates through COMPRE in complex, nonlinear ways. Control is not 
trivial, and ‘Tweaking the model to get the desired results” frequently backfires. To compare 
recommended actions resulting from COMPRE use with actual results, accelerated quality control 
techniques for hypothesis testing are being used. The null hypothesis (provided below) is that the 
ratio of programmatic risk to payoff is less than a benchmark value. 


H — < 
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A conclusion of the alternative hypothesis, that the normalized risk is greater than a prescribed 
standard, is a strong indicator for change. Improvement, as the development program evolves, is 
characterized by a reduction in this “normalized risk.” Benchmark values for this ratio are usually 
near 1 .0. A value of 1 .0 implies that the program management for the system development is 
using information (and resources) in such a way as to increase technological payoff (at the system 
level) by one unit for every additional unit of programmatic risk incurred. 

A method for monitoring the performance is shown in Figure 9.0-1. This plot shows the 
maximum likelihood ratio versus time or sample number. The maximum likelihood ratio is the 
ratio of the likelihood that the alternative hypothesis is true (program management is 
underperforming the benchmark) to the likelihood that the null hypothesis is true (program 
management is outperforming the benchmark). The greater the ratio, the more likely that 
management is poorly performing. The sample number is merely the number of quality 
monitorings that has occurred. Monitorings should be done frequently enough to allow for 
course corrections, but not too frequently to be an unnecessary burden by overemphasizing short- 
term performance. In this figure, sufficient data has been collected to determine 
underperformance of management, with respect to its benchmark. 
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Drawing a Ckality Conclusion Using Corrptetity Analysis 



Figure 9.0-1. Quality Monitoring Using COMPRE 


10.0 Final Assessment 

Figure 7.0-7 gives us an indication of the possible utility of COMPRE analysis on programmatic 
considerations. Specifically, a 20% decrease in schedule duration results in a 25% decrease in 
programmatic risk. Moreover, since schedule contractions are often associated with reductions in 
budgetary risk (and absolute costs), the impact may be even more favorable than depicted. Since 
COMPRE plots such as this one can be updated over time as more accurate information becomes 
available, the COMPRE methodology provides a mechanism for program management to 
dynamically tailor their project characteristics to acceptable risks and technological requirements. 
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Appendix A: Basis Function Formulas and Covariance Structures 


The general formulas for evaluating the technological maturities and programmatic risk are 
provided in Section 4.0, with the overall methodology provided in Section 5.0 This appendix 
provides the specific basis function manipulations to achieve system (and subsystem) level results. 

The six basis functions, corresponding to the six technological maturities, are given by: 


r <,(0 = t 

, , t t 1 

r. (/) = — + — 

1 2 2 

. , 3 t 1 e ' 

r 3(0 = — -+-J 

4 4 

. , t 2 e' 

r,{t) = — + — 

4W 2 2 

r s ( 0 = e ‘ 


Note that the basis functions for technological maturities of 1, 3, and 4 are linear combinations of 
the basis functions for technological maturities of 0, 2, and 5. The expected technological payoff 
for the standard basis functions may be found by integration to be: 


= y 

The other basis function expectations may be calculated as linear functions of these expectations: 

^ 4(0] = M + M 
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The standard basis covariances amongst themselves may be found through integration as: 


Q° ( 0- — 

4/ 4 
45 


C 22 (0 = 


C 5S (0 = ^' ( ^-^ 


c 02 (0 = 


12 


. 2„1 1 . 


c M (o = «"(-—) 

C 2J (0 = ^(^--2 + t) 


Finally, covariances amongst the other (nonstandard) basis functions are evaluated as linear 
combinations of the standard basis covariances, as given below. 


CM = 


CM , cm 




CM = 


3C„(0 . C„< r) 


C„<0 = 


C«(0 , c„(0 


C Cfi c <»('> I c ^<'> I £a(£) 

1 1 * ' ^ /-I A 


c i2 (0 = — (/) 1 Cz2(0 


• + - 


c rn r 301(0 , 3C 22(0 , CM , 0,(0 
uV ' 8 8 8 8 


C I4 (/) = 


_ cm . Q 2 (0 . CM . CM 




+- 


c „(0 = % (0 - c ” <0 


■ + - 




c 24 (/) 


c 22 (/) , cm 


+- 


26 



c 33 (o = 


9C 22 (t) | 3C a (Q t QsiO 

16 8 16 


C 34 ( t) = 
0,(0 = 


3C 22 (/) t C 25 ( Q | C„(/) 
8 2 8 

3Qj(0 , C„(0 
4 4 


0,(0 = 


c 22 (0 | c 25 (/) | C 55 (/) 

4 2 4 


C 4J (0 = 


Q ,(0 , C M (0 
2 2 


Appendix B: Numerical Example Using AXAF-I Example Data 

As an example of the use of the equations provided in Appendix A, a portion of the AXAF-I 
analysis is included in this appendix. Specifically, the development of the complexity results for 
the ISIM subsystem is provided, because it includes examples of all possible COMPRE 
calculations. The reader should refer to Section 4.0 for the general equations. Appendix A for the 
specific equations, and Figure 7.0-1 for the raw input data for the AXAF-I example. The 
development schedule duration is 5 years (t = 5). 

The expected technological payoff for the SIM component (technological maturity = 3) is given in 
Appendix A as: 


E[r mi (5)] = E[r i (5)] = = (3(5 2 / 3) + e 5 / 5) / 4 = 1 3.67 

4 4 

Similarly, the expected technological payoffs for the HRC and ACIS components of the ISIM are 
given as: 


E[r mc { 5)] = E[r, (5)] = = «5 2 / 3) + c 5 / 5) / 2 = 19.01 

E[r Aas ( 5)] = £[r 5 (5)] = e 5 /5 = 29.68 

Then, the composite development maturity for the ISIM system is given in Section 4.0 (also, see 
Table 7.0-1 for the relative budget weights) as: 

X ISM (S) = 028(13.67) + 022(19.01) + 050(29.68) = 22.85 

Note, that the composite technological maturity of the ISIM subsystem corresponds to 
approximately a 4.4. The covariance between the SIM and HRC is given in Appendix A as: 
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An organizational dispersion factor of 2 is applied to account for the cross-organization interface 
between these two components. This brings the covariance up to 629.32. The covariance 
between the SIM and ACIS is handled similarly as: 


C 




V(2(5-2 + 2) 

3 5 


, 215 '/ 


2(5) 


■?> 


= 52333 


An organizational dispersion factor of 2 is applied to account for the cross-organization interface 
between these two components. This brings the covariance up to 1046.66. Note that, due to a 
lack of interaction, the covariance between the HRC and ACIS components is 0. The covariances 
(variances) between the SIM and itself, HRC and itself, and ACIS and itself are 210.32, 486.80, 
and 1321.59, respectively. The overall programmatic risk for the ISIM subsystem is given in 
Section 4.0 as. 

cr /a.v/(5) = {0.28(0.28(2 10.32) + 0.22(629.32) + 0.50(1046.66)] + 0.22(0.28(629.32) + 0.22(486.80) + 0.50(0)] + 
0.50(0.28(1046.66) + 022(0) + 0.50(1 32 1 .59)] } 1/2 = 27.22 
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